Experimental Evaluation of Individualized Treatment Rules

The increasing availability of individual-level data has led to numerous applications individualized (or personalized) treatment rules (ITRs). Policy makers often wish empirically evaluate ITRs and compare their relative performance before implementing them in a target population. We propose new evaluation metric, the population average prescriptive effect (PAPE). PAPE compares ITR with that non-individualized rule, which randomly treats same proportion units. Averaging over range budget constraints yields our second area under curve (AUPEC). AUPEC represents an overall measure for evaluation, like receiver operating characteristic (AUROC) does classification, is generalization QINI coefficient utilized uplift modeling. use Neyman's repeated sampling framework estimate derive exact finite-sample variances based on random units assignment treatment. extend methodology common setting, experimental used both ITRs. In this case, variance calculation incorporates additional uncertainty due splits cross-validation. proposed metrics can be estimated without requiring modeling assumptions, asymptotic approximation, or resampling methods. As result, it applicable any including those complex machine learning algorithms. open-source software package available methodology.